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Abstract 


We  have  crossed  a  threshold  where  most  of  our  large  software  systems  can  no  longer  be 
constructed  as  monoliths  specified  by  a  single,  focused,  and  unified  team;  implemented  as  a 
unit;  and  tested  to  be  within  known  performance  limits.  They  are  now  constructed  as  groups 
of  interoperating  systems  (as  systems  of  systems)  developed  by  different  but  sometimes 
related  teams  and  made  to  interoperate  through  various  forms  of  interfaces.  Unfortunately, 
while  we  can  easily  conceive  these  large  systems  of  systems,  we  have  trouble  building  them. 
Software  engineering  practices  have  not  kept  pace,  and  the  problem  will  only  get  worse  as 
the  community  begins  to  build  Internet-scale  systems  of  systems  like  the  Global  Information 
Grid. 

This  technical  note  introduces  the  System-of-Systems  Navigator  (SoS  Navigator),  the 
collection  and  codification  of  essential  practices  for  building  large-scale  systems  of  systems. 
These  practices  have  been  identified  through  the  work  of  the  Integration  of  Software- 
Intensive  Systems  Initiative  at  the  Carnegie  Mellon  Software  Engineering  Institute.  SoS 
Navigator  provides  tools  and  techniques  to  characterize  organizational,  technical,  and 
operational  enablers  and  barriers  to  success  in  a  system  of  systems;  identify  improvement 
strategies;  and  pilot  and  institutionalize  these  strategies. 
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1  Introduction 


We  have  crossed  a  threshold  where  most  of  our  large  software  systems  can  no  longer  be 
constructed  as  monoliths  specified  by  a  single,  focused,  and  unified  team;  implemented  as  a 
unit;  and  tested  to  be  within  known  performance  limits.  They  are  now  constructed  as  groups 
of  interoperating  systems  (systems  of  systems)  developed  by  different  but  sometimes  related 
teams  and  made  to  interoperate  through  various  forms  of  interfaces. 

However,  we  have  reached  limits  in  our  ability  to  build  even  moderately  large  and 
interrelated  systems  of  systems  using  traditional  engineering  practices.  The  size  and 
complexity  of  those  systems,  the  number  of  people  they  involve,  and  the  difficulty  in 
specifying  what  the  interrelated  systems  are  supposed  to  do  and  how  they  are  to  do  it  lead  to 
almost  ubiquitous  cost  overruns — and  frequently  complete  failures.  Furthermore,  the  systems 
of  systems  that  are  built  tend  to  be  inflexible  and  difficult  to  maintain.  Also,  they  are  subject 
to  change  due  to  complex  business  rules  that  are  tightly  interconnected  with  the  rest  of  the 
application.  For  example,  it  is  noted  in  System  of  Systems  Interoperability  (SOSI):  Final 
Report  that  even  when  systems  can  be  interconnected,  the  connections  frequently  break  down 
as  new  versions  of  individual  systems  are  constructed  [Morris  04]. 

Large-scale  systems  of  systems  such  as  the  U.S.  Army’s  Future  Combat  Systems  (FCS)  are 
proving  to  be  very  expensive  and  difficult  to  build  [Francis  04].  Also,  a  number  of  planned 
Internet-scale  capabilities  represent  a  step  beyond  even  today’s  most  complex  systems  of 
systems.  For  example,  the  U.S.  Department  of  Defense  (DoD)  is  beginning  to  build  the 
Global  Information  Grid  (  GIG),  which  is  the  globally  interconnected  set  of  information 
capabilities,  processes,  and  personnel  that  support  the  vision  of  network-centric  warfare 
(NCW)  [GPO  00].  The  Department  of  Homeland  Security  (DHS)  is  investigating  ways  to 
provide  systems  that  connect  military,  civil  security,  medical,  and  other  agencies  to  provide 
coordinated  response  in  the  event  of  a  terrorist  attack.  The  medical  community  wishes  to 
connect  doctors,  hospitals,  insurance  companies,  pharmacies,  and  laboratories  into  networks 
that  provide  the  means  to  reduce  the  cost  and  improve  the  quality  of  medical  care  [CSI  05, 
HHS  05]. 

The  problems  manifest  with  moderately  large  systems  today  will  only  get  worse  as  we  move 
toward  large-  and  Internet-scale  systems  of  systems.  Research  is  underway  to  develop  new 
approaches  for  building  and  acquiring  systems  of  systems.  For  example,  the  Taiga  research 
project  at  Brown  University  focuses  on  Internet-scale  computing  and  is  finding  that  its 
complexity  represents  a  “new  reality  [that]  will  require  us  to  change  the  way  we  think  about 
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programs  and  programming.”1  In  2003,  Carnegie  Mellon  University  offered  a  graduate 
course  titled  Internet-Scale  Sensor  Systems:  Design  and  Policy.2  The  Workshop  on  Internet- 
Scale  Software  Technologies  (TWIST  99)  aims  to  gather  participants  from  industry  and 
academia  that  are  researching  or  developing  software  technologies  that  scale  to  the  Internet.3 
The  Carnegie  Mellon®  Software  Engineering  Institute  (SEI)  is  currently  completing  a  study 
to  characterize  ultra-large  systems  (ULS)  and  their  technical  and  research  challenges. 

It  is  important  now  to  initiate  practices  that  help  us  build  the  large  systems  of  systems  of  the 
near  and  intermediate  future  and  anticipate  those  necessary  for  the  Internet-scale  systems  and 
ULS  of  the  future.  Those  practices  need  to  allow  each  component  to  be  independently 
specified,  designed,  and  built — as  long  as  the  constituents  embrace  the  protocols,  standards, 
and  conventions  deemed  necessary  for  interoperation.  What  makes  these  practices  so  critical 
is  that  these  near-term  systems  of  systems  will  become  major  components  of  Internet-scale 
systems  in  the  future,  since  it  will  be  economically  infeasible  to  start  over  and  build  all  new 
components. 

This  technical  note  introduces  the  System-of-Systems  Navigator  (SoS  Navigator),  an 
integrated  set  of  practices  that  address  the  challenges  related  to  achieving  effective 
interoperability  in  a  systems-of-systems  context.  SoS  Navigator  is  a  result  of  the  work  of  the 
Integration  of  Software-Intensive  Systems  (ISIS)  Initiative  at  the  SEI.  ISIS  is  currently 
working  with  the  largest  systems  of  systems  of  today  and  the  intermediate-term  future  to 
identify,  mature,  and  transition  software  engineering  methods  and  techniques  that  enable 
organizations  to  integrate  components,  systems,  and  systems  of  systems.  4  Team  members  of 
ISIS  also  provide  guidance  on  the  selection  and  use  of  technologies  and  methods  to  develop, 
implement,  and  evolve  interoperable  systems. 


1 .1  Overview  of  This  Technical  Note 

Section  2  provides  an  overview  of  SoS  Navigator  and  its  structure.  Section  3  describes  the 
foundational  element,  the  SoS  Framework.  Section  4  summarizes  the  Charting  elements, 
called  SoS  Diagnose  and  SoS  Analyze.  Section  5  describes  the  Improving  elements,  named 
SoS  Pilot  and  SoS  Guide.  Section  6  provides  closing  remarks. 


1  For  more  information  on  this  project,  go  to  http://www.cs.brown.edu/research/projects/taiga.html. 

2  For  more  information,  visit  http://www.cs.cmu.edu/~srini/15-829A/. 

3  For  more  details,  go  to  http://www.isr.uci.edu/events/twist/twist99/. 

®  Carnegie  Mellon  is  registered  in  the  U.S.  Patent  and  Trademark  Office  by  Carnegie  Mellon 
University. 

4  ISIS  has  been  involved  in  systems-of-systems  efforts  such  as  FCS,  federal  and  commercial  medical 
systems,  satellite  constellations,  and  the  GIG. 
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2  Purpose  and  Structure  of  the  SoS  Navigator 


SoS  Navigator  is  an  integrated  set  of  practices  that  address  the  challenges  related  to  achieving 
effective  interoperability  in  a  systems-of-systems  context.  SoS  Navigator  is  strongly 
influenced  by  the  belief  that  state-of-the-art  systems  of  systems  like  FCS  and  future  Internet- 
scale  systems  of  systems  like  the  GIG  are  approaching  or  will  reach  a  state  that  has  the 
following  characteristics: 

•  The  constituents5  of  such  large-scale  systems  will  be  so  highly  dynamic  that  the  systems 
of  systems  are  essentially  unbounded — constantly  growing  and  shrinking — with  no 
authority  having  complete  knowledge  of  all  of  the  parts.  This  circumstance  is  in  contrast 
to  a  closed  system  with  rigid,  impermeable  boundaries  and  normally  well-defined  control 
authorities. 

•  The  stakeholders  of  the  constituent  systems  will  be  highly  diverse  and  will  have 
motivations  that  compete  or  conflict  with  those  that  provided  the  impetus  for  the  effort  to 
build  the  system  of  systems. 

•  There  will  be  no  clear  development,  acquisition,  and  operation  cycles  for  the  system  of 
systems  as  a  whole.  These  systems  of  systems  will  not  be  “built”  in  the  sense  that  a 
master  architect  envisions  the  parts  and  their  relationships;  rather  they  will  evolve  into 
existence  and  change  through  their  life  cycles  as  new  constituents  are  built,  existing 
systems  connect  to  become  constituents,  and  constituents  leave. 

SoS  Navigator  helps  organizations  chart  a  technical,  organizational,  and  operational  path 
through  the  system-of-systems  environment  and  prepare  for  the  even  more  demanding 
environments  of  the  future.  SoS  Navigator  is  composed  of  these  major  elements: 

•  SoS  Framework — codifies  core  paradigms,  principles,  processes,  and  techniques 
associated  with  effective  systems  of  systems 

•  SoS  Diagnose — identifies  and  characterizes  the  enablers  and  barriers  for  a  particular 
system  of  systems 

•  SoS  Analyze — applies  the  SoS  Framework  to  the  information  uncovered  during  the  SoS 
Diagnose  element  to  identify  improvement  strategies  for  the  system-of-systems  effort 

•  SoS  Pilot — demonstrates  and  prototypes  selected  systems-of-systems  practices  in  a 
particular  system-of-systems  context 

•  SoS  Guide — provides  mechanisms  for  institutionalizing  improvements  of  systems-of- 
systems  practices  across  the  stakeholders  of  a  system  of  systems 

5  For  our  purposes,  the  term  constituents  refers  to  autonomous  components  of  a  system  of  systems. 
Constituents  can  be  automated  or  mechanized  and  can  also  mean  individuals  or  organizations. 

When  referring  to  constituents  represented  in  graphs,  we  will  typically  call  them  nodes.  When  we 
intend  a  reference  to  individuals  or  organizations  only,  we  will  use  stakeholders. 
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As  shown  in  Figure  1,  the  SoS  Framework  element  provides  the  overall  foundation  for  the 
SoS  Navigator  structure  and  informs  each  of  the  other  elements.  Collectively,  the  SoS 
Diagnose  and  SoS  Analyze  elements  assist  organizations  to  chart  where  they  are  and 
determine  where  they  should  go.  Similarly,  SoS  Pilot  and  SoS  Guide  collectively  assist 
organizations  to  improve  their  system-of-systems  practices  based  on  the  results  of  that 
charting. 


Framework 

Figure  1:  SoS  Navigator  Structure 

The  elements  SoS  Diagnose,  SoS  Analyze,  SoS  Pilot,  and  SoS  Guide  are  applied  iteratively 
as  necessary  to  enhance  the  capabilities  of  system-of-systems  constituent  organizations. 
These  elements  can  be  used  at  any  stage  of  a  system  of  systems.  They  can  also  be  used 
periodically  as  needed  throughout  the  life  of  a  system  of  systems. 

The  SoS  Navigator  elements  are  explained  further  in  the  following  three  sections:  (1)  SoS 
Framework  in  Section  3,  (2)  Charting  (for  SoS  Diagnose  and  SoS  Analyze)  in  Section  4,  and 
(3)  Improving  (for  SoS  Pilot  and  SoS  Guide)  in  Section  5. 
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3  SoS  Framework 


The  SoS  Framework  element  is  a  diverse  set  of  core  paradigms  and  principles  along  with 
processes  and  techniques  that  the  ISIS  Initiative  has  developed  in  our  work  with 
organizations  attempting  to  build,  field,  and  evolve  systems  of  systems. 6  We  expect  to 
continue  refining  and  expanding  this  body  of  knowledge  through  our  collaborations  with 
system-of-systems  practitioners  and  other  research  organizations. 


3.1  Core  Paradigms  and  Principles 

Within  the  SoS  Framework  element,  information  associated  with  core  paradigms  and 
principles  can  be  organized  into  several  interrelated  categories.  These  categories  of 
information  include 

•  system-of-systems  context 

•  system-of-systems  perspectives 

•  system-of-systems  organizations 

•  unbounded  systems 

The  following  sections  identify  some  of  the  key  points  within  these  categories. 

3.1.1  System-of-Systems  Context 

Understanding  the  context  in  which  large-scale  systems  of  systems  are  built  and  operated  is 
essential  to  understanding  the  types  of  processes  and  techniques  that  will  be  successful.  The 
following  fundamental  statements  about  systems  of  systems  reveal  this  context. 

•  A  system  of  systems  is  more  than  “just”  hardware  and  software.  Systems  of  systems 
are  composed  of  various  types  of  constituents,  including  computing  components  (e.g., 
hardware/software  components),  organizations  that  build  and  operate  those  components, 
and  the  individual  people  within  those  organizations.  All  of  the  influences  arising  from 
the  interactions  among  those  constituents  must  be  considered — whether  from  common, 
competing,  and  conflicting  motivation  and  intent  or  differing  needs  and  expectations. 

•  A  system  of  systems  has  a  purpose,  even  when  individual  constituents  do  not  share  the 
overall  intent  of  the  people  or  organizations  that  conceived  it.  In  fact,  it  is  common  for 
some  individual  constituents  not  to  share  in  the  overall  intent,  yet  still  participate  in  a 
system  of  systems.  Perhaps  the  most  common  example  of  this  aspect  is  the  commercial 


6  We  use  the  term  framework  with  considerable  caution  because  it  has  several  different  meanings. 
Our  definition  is  intended  only  in  reference  to  SoS  Navigator. 
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off-the-shelf  (COTS)  component  that  provides  a  capability  to  many  systems  of  systems 
but  does  not  share  in  the  intent  of  any  those  systems.  As  Internet-scale  systems  of 
systems  become  possible,  they  will  likely  incorporate  many  constituents  that  do  not  share 
the  overall  intent. 

•  A  system  of  systems  is  built  to  operate  within  a  specific  cultural,  business,  and  legal 
environment.  This  circumstance  constrains  how  and  why  the  system  of  systems  is  built 
and  determines  the  types  of  constituents  that  can  be  involved.  It  often  also  imposes 
restrictions  (e.g.,  security).  However,  large-scale  systems  of  systems  must  frequently 
incorporate  constituents  that  do  not  share  all  elements  of  the  environment. 

•  A  system  of  systems  is  subject  to  constantly  changing  needs,  technologies,  and 
environment.  In  a  system  of  systems,  there  is  a  need  to  explicitly  address  uncertainty 
and  unanticipated  changes,  since  the  requirements  of  users,  the  environment  in  which  the 
system  of  systems  operates,  and  the  technologies  that  populate  the  system  of  systems  are 
bound  to  change  over  time. 

•  Adaptability  and  scalability  are  key  requirements.  To  a  much  greater  extent  than  for 
monolithic  systems,  there  is  a  need  for  systems  of  systems  to  be  scalable  in  a  cost- 
effective  fashion  and  adapt  to  the  changing  circumstances.  Continual  evolution  is  a 
characteristic  of  system  of  systems  [Carney  05].  The  nature  and  rate  of  evolution  of  the 
interoperability  relationship  are  critical. 

3.1.2  System-of-Systems  Perspectives 

The  multiple  organizations  involved  in  systems  of  systems  will  have  different  perspectives  on 
technology,  organizational  relationships,  commitments,  and  other  factors.  One  organization 
may  be  unaware  of  the  perspectives  of  some  other  organizations — or  even  of  their  existence. 

To  achieve  interoperation  in  a  system-of-systems  context,  a  diverse  set  of  stakeholders  across 
many  organizations  must  coordinate  efforts  and  recognize  multiple  perspectives — in 
management,  assembly,  and  operations — as  detailed  below: 

•  management 

This  perspective  on  interoperability  is  concerned  with  issues  such  as  schedules,  risk 
management,  supplier  coordination  through  contracts,  motivation,  incentives,  and 
teamwork. 

•  assembly 

This  perspective  is  concerned  with  system  and  software  development,  maintenance,  and 
evolution  activities,  such  as  forming  the  architecture,  testing  and  integrating  systems 
effectively,  and  transitioning  systems  to  the  user  community. 

•  operations 

This  perspective  is  concerned  with  activities  performed  by  the  end  user  in  the  actual 
operation  of  the  system. 

The  complex  relationships  of  the  critical  stakeholders  across  those  perspectives  need  to  be 
managed  throughout  the  lifetime  of  the  system  of  systems. 
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3.1.3  Organizational  Entities 

In  addition  to  differing  perspectives,  systems  of  systems  involve  several  distinct 
organizational  entities. 

•  global  system-of-systems  entity 

In  any  system  of  systems,  there  is  an  entity  that  represents  its  global  capability.  In  some 
cases,  this  entity  is  an  actual  program.  For  example,  FCS  is  a  program  of  record  trying  to 
construct  a  system  of  systems.  Also,  in  the  Joint  Battle  Management  Command  and 
Control  System,  the  entity  exists  (i.e.,  U.S.  Joint  Forces  Command  [USJFCOM])  but 
does  not  have  managerial  control  over  all  the  constituent  systems.  A  global  system-of- 
systems  entity  exists  even  in  cases  where  the  system  of  systems  is  dynamically  composed 
of  constituents  that  are  unaware  of  the  existence  of  the  system  of  systems.  The  global 
entity  is  responsible  for  the  composition  of  the  constituents. 

•  autonomous  (or  semiautonomous)  constituents 

Constituents  in  a  system  of  systems  often  operate  with  an  unexpected  degree  of 
independence.  This  independence  is  sometimes  the  result  of  the  distinct  acquisition  and 
management  authorities  for  the  constituents.  The  complexity  of  many  systems  of  systems 
and  the  competing  demands  of  independent  and  system-of-systems  operation  lead  to 
surprisingly  independent  constituents,  even  when  they  are  under  the  managerial  control 
of  a  single  entity.  Each  constituent  often  evolves  independently  from  other  constituents, 
but  it  contributes  to  the  overall  evolution  of  the  system  of  systems. 

•  communities  of  interest  (COIs) 

Stakeholders  responsible  for  constituents  in  large-scale  (and  in  the  future,  for  Internet- 
scale)  systems  of  systems  often  find  advantages  in  forming  alliances  of  common  interest. 
Those  alliances  tend  be  relatively  long-lived,  such  as  an  alliance  of  command  and  control 
interests  working  together  to  establish  common  interfaces. 

•  neighborhoods 

Related  to  COIs,  but  more  dynamic,  are  neighborhoods,  which  reflect  that  constituents 
must  interact  for  a  given  purpose.  At  any  given  time,  an  individual  constituent  will  be  a 
member  of  many  neighborhoods.  For  example,  one  neighborhood  may  include  other 
constituents  with  related  schedule  dependencies.  Another  neighborhood  may  be  based  on 
a  shared  technical  decision.  A  third  sort  of  neighborhood  may  be  based  on  execution 
pathways.  In  dynamically  composed  systems  of  systems,  such  neighborhoods  may  exist 
only  for  a  specific  execution  sequence. 

Sometimes  these  entities  are  aligned  for  a  common  purpose.  More  often,  their  purposes  are 
misaligned  to  some  degree. 

3.1.4  Unbounded  Systems 

Some  of  today’s  large  systems  of  systems  (like  FCS)  and  all  Internet-scale  systems  of 
systems  of  the  future  are  examples  of  unbounded  systems,  in  which  an  unknown  number  of 
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participants  might  interact  at  any  given  instance.  Unboundedness  leads  to  consequences  such 

as 

•  emergence 

Unbounded  systems  display  behaviors  that  differ  from,  and  are  not  easily  foreseen  as 
arising  from,  the  collective  properties  of  the  constituents  that  make  up  the  system  of 
systems.  These  global  behaviors  emerge  from  the  cumulative  actions  and  interactions  of 
the  constituents  propagated  throughout  the  system  of  systems  and  can  have  a  positive  or 
negative  effect.  In  the  best  case,  emergent  properties  will  provide  unanticipated  benefits 
to  the  users  of  the  systems  of  systems.  For  example,  Internet  Protocol  routing  is 
surprisingly  resilient,  due  to  the  emergence  of  unanticipated  pathways  for 
communication.  In  the  worst  cases,  emergent  properties  will  destroy  system-of-systems 
capabilities,  as  can  occur  when  circuits  reverberate  with  information  repeatedly  passed 
around  the  Internet. 

•  limited  visibility 

No  individual  constituent  perceives  the  entire  system.  Instead,  all  constituents  must  deal 
with  incomplete  information  about  the  other  participants  and  the  current  operational 
situation. 

•  inadequacy  of  traditional  systems  engineering  approaches 

Many  traditional  approaches  will  not  work,  in  fact.  Tight  coupling  among  systems, 
central  process  and  data  control,  hierarchical  architectures,  and  top-down  enforcement  of 
all-encompassing  standards — all  are  unlikely  to  lead  to  capable  and  flexible  systems  of 
systems.  Those  approaches  were  developed  for  a  context  in  which  the  development  and 
evolution  of  constituents  were  controlled  by  a  shared  management. 

Algorithms  for  unbounded  systems  are  being  identified.  New  methods  include 

•  ways  to  take  advantage  of  emergent  properties  by  encouraging  cooperation  without  tight 
or  inflexible  coordination 

•  dynamic  adaptation  to  changing  circumstances 

•  means  to  identify  and  capitalize  on  opportunistic  interactions 

•  anticipatory  assistance  to  neighbors 

•  built-in  resilience  and  redundancy  in  architectures 


3.2  Processes  and  Techniques 

As  noted  in  the  preceding  sections,  many  traditional  engineering  and  management  processes, 
methods,  and  techniques  do  not  scale  and  are  often  inadequate  for  today’s  system  of 
systems — let  alone  for  the  Internet-scale  systems  of  tomorrow.  Driven  by  a  model  of 
“thinking  globally,  yet  acting  locally,”  we  need  new  paradigms  that  encompass  practices  in 
the  following  situations: 

•  No  one  stakeholder  group  or  individual  can  have  complete  system-of-systems  insight. 

•  Central  control  has  limited  effectiveness;  distributed  control  is  essential. 
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•  System-of-systems  capabilities  and  properties  emerge  from  cumulative,  indirect  effects 
of  local  actions  and  neighbor  interactions. 

•  Broader  sets  of  stakeholders,  including  users,  must  be  directly  involved  throughout  the 
life  of  a  system  of  systems. 

•  Local  decisions  and  reward  systems  must  be  tempered  by  understanding  of  system-of- 
systems  purpose  and  goals. 

A  central  aspect  to  engineering  and  managing  in  a  system-of-systems  environment  is  an 
understanding  of  the  relationships  among  the  various  constituents.  The  SoS  Framework 
element  includes  a  set  of  interoperability  maps — graphs  to  capture  information  associated 
with  these  relationships.  Each  of  the  other  SoS  Navigator  elements  (i.e.,  Diagnose,  Analyze, 
Pilot,  and  Guide)  can  use  this  information.  The  SoS  Framework  element  also  includes  a  set  of 
processes  to  create,  use,  and  refine  this  relationship  information. 

The  following  sections  describe  interoperability  maps  and  their  associated  processes. 

3.2.1  Interoperability  Maps 

Interoperability  maps  characterize  the  relationships  in  a  system  of  systems  from  three 
perspectives:  (1)  the  global  system-of-systems  entity,  (2)  individual  constituents  (represented 
as  nodes  in  the  graph),  and  (3)  individual  agreements  (represented  as  arcs  between  a  pair  of 
nodes). 

These  maps  permit  the  capture  of  information  about  how  constituents  actually  influence  one 
another  in  a  system  of  systems.  That  is,  they  portray  the  reality  of  how  things  work — not  how 
they  are  supposed  to  work — and  represent  actual  understandings,  intents,  and  expectations  of 
constituents,  as  opposed  to  what  is  stated  in  acquisition  and  design  artifacts. 

There  are  three  forms  of  interoperability  maps  currently  in  the  SoS  Framework  element. 

1.  Context  Interoperability  Map  depicts  high-level,  system-of-systems-wide  information 
about  contractual,  funding,  requirements,  hardware,  oversight,  and  build/integrate 
influence  relationships  from  the  global  system-of-systems  entity  perspective. 

2.  Node-centric  Interoperability  Map  portrays  information  about  contractual,  funding, 
requirements,  hardware,  oversight,  and  build/integrate  influence  relationships  from  the 
perspective  of  individual  constituents. 

3.  Arc-centric  Interoperability  Map  represents  information  about  needs,  offers, 
expectations,  intentions,  and  negotiated  agreements  between  the  constituents  involved  in 
the  influence  relationship. 
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Context  Interoperability  Map 

The  nodes  represented  in  the  Context  Interoperability  Map  are 


•  systems 

•  major  management  entities  (e.g.,  contractors,  program  offices,  or  agencies) 

•  funding  organizations  (e.g.,  appropriations  committees) 

•  oversight  organizations  (e.g.,  regulatory  boards  or  standards  bodies) 

•  contractual  organizations 

As  the  Context  Interoperability  Map  presented  in  Figure  2  illustrates,  arcs  connect  nodes  that 
have  an  influence  relationship.7  These  influence  relationships  can  be  highly  complex, 
encompassing  multiple  dimensions  of  schedule,  contracting,  and  performance.  The  Context 
Interoperability  Map  conveys  a  general  “lay  of  the  land”  and  may  also  provide  insight  into 
possible  areas  of  the  system  of  systems  that  would  be  good  candidates  for  further  exploration. 


— Funding -  - Oversight - 

-Requirements -  Build/Integrate— 


Figure  2:  Context  Interoperability  Map 


1  The  interoperability  maps  shown  in  this  section  are  conceptual  models  designed  to  illustrate  the 
kinds  of  maps  produced  in  the  SoS  Framework  element.  They  do  not  reflect  an  actual  system  or 
system  effort. 
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The  Context  Interoperability  Map  allows  the  SoS  Navigator  team  to  capture  the  broad 
influences  on  the  system  of  systems.  In  effect,  this  graph  represents  the  viewpoint  of  the 
system-of-systems  global  entity  responsible  for  the  overall  system  of  systems.  It  identifies 
and  documents  many  individual  constituents  that  participate  in  the  systems-of-systems  effort. 
However,  it  does  not  attempt  to  identify  all  of  the  influences  that  impinge  on  individual 
nodes;  that  is  the  function  of  the  Node-centric  Interoperability  Map. 

Node-Centric  interoperability  Map 

From  the  standpoint  of  a  constituent,  the  Node-centric  Interoperability  Map  (shown  in  Figure 
3)  documents  the  influences  in  a  system  of  systems. 


-Requirements -  -Buld/lntegrate 


Figure  3:  Node-centric  Interoperability  Map 

Node-centric  Interoperability  Maps  are  specialized  to  the  perspective  of  a  single  program 
management  office,  contractor,  or  other  type  of  constituent.  They  reveal  what  is  “visible”  to 
the  constituent.  An  important  aspect  is  that  a  constituent  represents  the  relevant  interests  of 
“downstream”  constituents  to  an  “upstream”  constituent.  For  example,  in  Figure  3,  Program 
Office  “C”  would  represent  any  schedule  constraints  that  it  has  with  any  downstream 
constituents  (Agency  “Y”  and  Prime  Contractor  “C”)  in  its  schedule  relationship  with 
Program  Office  “A.”  This  notion  of  pass-through  or  transitive  influences  allows  the  influence 
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relationships  affecting  a  particular  constituent  to  be  understood  without  requiring  that 
constituent  to  have  insight  into  the  entire  system  of  systems. 

Figure  3  shows  how  influence  relationships  can  be  fairly  complex: 

•  The  direction  of  an  arc  represents  the  primary  direction  of  influence. 

•  The  destination  node  of  each  arc  (i.e.,  the  “upstream”  constituent)  has  a  need  that 
represents  the  claimed  minimal  set  of  critical  expectations  from  the  source  node  of  the 
arc  (possibly  as  function  of  schedule,  value,  or  quality). 

•  The  arc  source  node  has  an  offer  that  represents  the  broadest  set  of  relevant  things  that  it 
can  feasibly  provide  to  the  destination  node. 

•  Each  arc  has  an  associated  agreement  that  may  be  in  part  implicit,  informal,  or  tacit. 

-  Agreements  derive  from  negotiation — often  informal — of  needs  and  offers. 

-  Agreements  may  be  vague  initially  and  then  refined  as  detail  is  needed  and  understood. 

-  In  combination  with  the  context  in  which  the  neighbors  operate  and  the  trust  they  place 
in  their  partners,  agreements  determine  the  intents  and  expectations  along  each  arc. 

Node-centric  Interoperability  Maps  provide  a  mechanism  to  establish  consistency  between 
what  one  constituent  believes  to  be  important  interrelationships  (as  reflected  in  the  Context 
Interoperability  Map)  and  what  other  constituents  believe  to  be  important. 

In  addition  to  providing  sufficient  detail  to  support  the  analysis  of  inconsistencies  and 
conflicts,  the  Node-centric  Interoperability  Maps  identify  relationships  to  organizations 
outside  of  the  purview  of  a  global  system-of-systems  entity.  For  example,  Figure  3  represents 
relationships  between  Program  Office  “A”  and  several  constituents  not  normally  under  the 
purview  of  most  global  system-of-systems  entities  (i.e.,  appropriators,  authorizers,  and 
regulatory  oversight  bodies).  Notice  that  these  constituents  can  have  a  significant  impact  on  a 
system  of  systems  but  often  are  not  considered. 

Arc-Centric  Interoperability  Map 

Arc-centric  Interoperability  Maps  express  and  make  explicit  the  (often  implicit)  assumptions 
that  go  into  an  influence  relationship.  They  can  be  used  in  situations  where  influence 
relationships  are  particularly  complex,  critical,  or  easily  misunderstood.  In  an  Arc-centric 
Interoperability  Map,  as  Figure  4  demonstrates,  the  needs  of  the  requesting  constituent  are 
expressed  as  a  set  of  minimum  critical  needs  (MCNs) — the  absolute  minimum  that  is  truly 
necessary  to  satisfy  the  requestor’s  constraints.  The  response  from  the  offering  constituent  is 
expressed  as  a  set  of  broadest  feasible  offers  (BFOs) — the  “most  generous”  response  it  can 
provide  that  does  not  violate  its  constraints. 
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Minimum  Critical  Needs 


Negotiated  Agreement 


Broadest  Feasible  Offers 


(MCNs) 


(BFOs) 


Needed-by  date 
(rango) 

Product  quality 
(e.g.,  error  density) 
Contingent  cost 
liability 

(remediation  cost  if 
late,  poor  quality, 
etc.) 
etc. 


Delivery  date  (range) 

Acceptance  testing  with  specified  quality  attributes  (e  g., 
latent  defect  rate,  etc.) 

Information-sharing  between  program  offices  (e  g  .  cross- 
attendance  at  program  reviews,  selected  internal  reviews, 
etc.) 


\ 


\ 


Promised  delivery 
date  (range) 


Unit-level  testing 
Advanced  warning 
if  delay  anticipated 
etc. 


needs 


wants 


/ 


dependency 


Figure  4:  Arc-centric  Interoperability  Map 

Where  there  is  an  overlap  between  the  MCNs  and  BFOs,  an  agreement  is  possible;  where 
there  is  no  overlap,  no  feasible  match  between  the  requestor’s  needs  and  the  offering 
constituent’s  capabilities  exists.  In  short,  no  overlap — even  after  negotiating  (i.e.,  exploring 
whether  restating  needs  and  offers  can  possibly  result  in  an  overlap) — indicates  that  no 
agreement  is  possible.  The  focus  of  Arc-centric  Interoperability  Maps  on  MCNs  and  BFOs  is 
important,  because  those  assumptions  represent  the  end  points  of  a  range  within  which  a 
negotiated  agreement  is  possible.  Interestingly,  these  end  points  are  often  not  the  same  as  the 
negotiated  agreement,  since  the  agreement  often  represents  a  more  optimistic  view  of  events. 

3.2.2  Processes  for  Managing  Interoperability  Relationships 

The  SoS  Framework  element  currently  includes  these  processes  to  manage  the  various 
relationships  among  constituents  for  a  given  system  of  systems: 

•  Form  and  evolve  agreements  among  constituents.  This  process  begins  with  identifying 
and  characterizing  MCNs  and  BFOs.  Through  negotiation,  common  ground  is  identified. 

•  Propagate  constraints  for  needs  and  offers.  As  parties  negotiate  and  form  MCNs  and 
BFOs,  they  act  as  a  proxy  for  their  “downstream”  constituents.  This  approach  provides  a 
mechanism  to  work  more  effectively  within  the  nature  of  unbounded  systems. 

•  Manage  intentions  and  expectations  informed  by  trust.  Trust  reflects  confidence  in 
the  information  contained  in  MCNs  and  BFOs.  Divergent  expectations  and  intentions 
may  cause  an  interoperability  problem. 
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4  Charting 


In  our  experience,  many  system-of-systems  efforts  fail  because  organizations  either  are 
unaware  of  the  existence  of  the  paradigms  and  principles  reflected  in  the  SoS  Framework 
element  or  do  not  understand  their  implications.  For  large-  or  Internet-scale  system-of- 
systems  efforts  to  succeed,  multiple  “world  views”  must  be  understood  and  addressed  so  that 

•  the  differing  and  often  competing  interests  of  the  system-of-systems  entities  (global 
system-of-systems  entity,  autonomous  constituents,  COIs,  and  neighborhoods)  can  be 
aligned  where  appropriate  and  accommodated  where  necessary 

•  where  competing  interests  are  not  well  recognized,  the  fewest  number  of  constraints 
possible  can  be  imposed  on  constituents,  COIs,  and  neighborhoods 

It  is  not  necessary  to  ensure  that  all  entities  are  in  lockstep.  Rather,  the  need  is  to  help 
organizations  know  when  they  have  to  fall  in  step  and  when  they  have  freedom  to  vary.  Thus, 
aligning  world  views  will  likely  also  involve  agreeing  on  what  not  to  constrain.  We  believe 
that  extracting  and  understanding  the  multiple  organizational,  technical,  and  operational 
perspectives — and  their  interrelationships — helps  to  effectively 

•  determine  where  those  relationships  are  aligned  or  unaligned  across  the  different  system- 
of-systems  entities 

•  make  improvements  to  the  alignment  where  needed 

•  establish  the  minimal  agreements  that  are  necessary  for  participants  to  work  effectively 

SoS  Diagnose  and  SoS  Analyze,  the  Charting  elements,  provide  approaches  that  promote  the 
understanding  of  the  current  state  of  a  particular  system  of  systems  and  determine  necessary 
improvements.  The  primary  objectives  of  these  elements  are  to  provide  stakeholders  with  (1) 
a  profile  of  the  constituents  and  their  primary  patterns  of  relationships,  barriers,  and  enablers 
to  achieving  the  desired  system-of-systems  interoperation  and  (2)  recommended  actions  to 
enhance  the  likelihood  of  success.  The  Charting  Team,  composed  of  two  to  four  members 
with  expertise  in  the  various  viewpoints  associated  with  systems  of  systems,  would  typically 
perform  the  SoS  Diagnose  and  SoS  Analyze  elements  sequentially. 


4.1  SoS  Diagnose 

The  SoS  Diagnose  element  is  a  set  of  techniques  designed  to  assist  organizations  with 
problems  in  program  management;  system  construction  and  maintenance;  or  operation  of 
large,  complex,  heterogeneous,  software-intensive  systems  of  systems.  The  SoS  Diagnose 
element  assists  organizations  in  the  identification  and  characterization  of  key  enablers  and 
banders  to  achieving  and  maintaining  required  levels  of  interoperability.  It  also  helps 
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organizations  gain  insight  into  the  key  relationships  forged,  information  exchanged,  and 
decisions  made  between  constituents. 

To  gather  information  about  the  constituents  and  the  most  important  relationships,  the  SoS 
Diagnose  element  calls  for  a  series  of  structured  discussion  sessions  and  workshops  with 
various  stakeholders  that  represent  the  diverse  perspectives  of  the  system  of  systems. 
Quantifiable  data  and  other  documented  information  are  used  to  augment  the  discussion  and 
workshop  sessions.  Interoperability  maps  are  an  additional  technique  used  to  elicit  and 
capture  information  relevant  to  management,  construction  and  assembly,  and  fielding  and 
operations. 

The  SoS  Diagnose  element  is  a  collaborative  engagement,  in  which  the  Charting  Team  leads 
various  stakeholders  associated  with  the  specific  system  of  systems  in  discussion  sessions 
and  workshops.  These  sessions  identify  issues,  explore  causes,  expand  and  validate 
interoperability  information  and  diagrams,  and  identify  existing  effective  practices. 

The  SoS  Diagnose  element  is  conducted  in  two  phases,  which  are  summarized  in  the 
following  sections:  4.1.1  (context  setting)  and  4.1.2  (data  gathering). 

4.1.1  Context-Setting  Phase 

In  the  context-setting  phase,  participants  agree  on  a  scope  that  will  provide  sufficient  value 
within  the  current  political,  cultural,  financial,  and  scheduling  realism  of  the  system  of 
systems  to  be  explored.  This  phase  legitimizes  the  Charting  Team  by  establishing  the  contract 
between  it  and  the  sponsoring  organization  and  creates  the  foundation  for  the  data-gathering 
phase  and,  later,  the  SoS  Analyze  element.  In  forming  the  scope,  team  members  construct 
realistic  expectations  of 

•  communities  and  organizations  that  will  participate 

•  process  to  be  used8 

•  result  to  be  achieved  (e.g.,  producing  a  set  of  recommendations)  at  the  completion  of  the 
companion  element,  SoS  Analyze 

During  the  context- setting  phase,  the  Charting  Team  seeks  to 

•  establish  an  overview  of  the  organizational  structures 

•  identify  the  key  players  and  roles 

•  discover  the  common  acronyms  and  culture  of  the  communities  and  organizations 
participating  in  the  system  of  systems 

•  assimilate  general  background  information  on  the  application  domain  (Where  possible, 
the  general  background  material  comes  from  existing  sources  readily  available  to  the 


The  determination  of  a  process  to  be  used  must  take  account  of  the  time-commitment  expectations 
of  the  participants. 
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sponsoring  organization,  such  as  technical  and  nontechnical  briefings  and  other  related 
documentation.) 

•  form  a  top-level  system-of-systems  overview 

•  determine  schedules 

•  understand  funding  sources  and  requirements 

The  team  also  collects  an  initial  set  of  the  interoperability  challenges  or  issues  facing  the 
organizations.  Often,  a  sponsoring  organization  already  has  such  a  list.  Time  and  disruption 
for  individuals  and  their  organizations  is  minimized  to  the  extent  practical. 

4.1.2  Data-Gathering  Phase 

The  goal  of  the  data-gathering  phase  is  to  execute  the  data  collection  process.  Materials 
provided  to  the  Charting  Team  as  part  of  the  SoS  Diagnose  element  include 

•  a  set  of  discussion  topics,  templates,  and  guidelines  for  organizing  the  discussion  and 
workshop  sessions 

•  tools  for  collecting  the  data  gathered 

•  guidelines  for  tailoring  this  phase  to  the  needs  of  a  specific  system-of-systems  context 
and  the  negotiated  scope 

The  discussion  and  workshop  sessions  involve  staff  members  who  are  responsible  for  or 
affected  by  the  various  elements  of  the  system  of  systems.  These  sessions  should  include  as 
many  different  perspectives  as  possible:  management,  development  and  maintenance,  and 
operations.  A  key  role  for  the  Charting  Team  during  the  sessions  is  to  moderate  and  focus  the 
discussion  on  issues  related  to  interoperability  and  systems  of  systems.  The  emphasis  during 
the  data-gathering  phase  is  on  understanding  what  is  transpiring  in  a  system-of-systems 
context;  it  is  not  a  process  assessment  or  a  technique  to  assign  blame. 

During  the  discussions  and  workshop  sessions,  draft  interoperability  maps  are  created,  vetted, 
and  revised.  The  Charting  Team  typically  creates  a  series  of  interoperability  maps  with 
multiple  levels  of  abstraction  and  detail.  The  maps  provide  a  graphical  representation  that 
facilitates  capturing,  understanding,  and  analyzing  relationships  among  constituents  of  the 
system  of  systems.  The  following  section  provides  several  examples  of  using  interoperability 
maps  as  part  of  the  SoS  Diagnose  element. 

4.1.3  Using  Interoperability  Maps 

A  Context  Interoperability  Map  is  initially  produced  by  the  Charting  Team  in  conjunction 
with  the  global  system-of  systems  entity.  During  the  various  discussion  and  workshop 
sessions,  that  Context  Interoperability  Map  is  reviewed  and  revised  as  necessary.  From  that 
map,  the  Charting  Team  identifies  constituents  that  require  further  exploration  through  Node¬ 
centric  Interoperability  Maps. 
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Draft  Node-centric  Interoperability  Maps  and/or  Arc-centric  Interoperability  Maps  are 
created  or  revised  during  relevant  sessions.  The  Charting  Team  facilitates  the  creating  and 
vetting  of  the  maps,  but  the  information  must  come  from  the  stakeholders — ideally,  system 
engineers  (for  the  system  of  systems  and  individual  constituents),  managers,  operational 
users,  financial  personnel,  and  contracts  experts. 

For  Arc-centric  Interoperability  Maps,  as  much  information  as  possible  is  identified  about 
needs,  offers,  agreements,  context,  level  of  trust,  intent,  and  expectations.  The  accuracy  of 
this  information  is  essential,  but  completeness  is  less  so.  In  other  words,  the  map  should 
contain  what  is  known,  but  no  assumptions  should  be  made  about  what  is  unknown. 
Gathering  this  information  may  require  interaction  with  partners  in  the  influence 
relationships. 


4.2  SoS  Analyze 

The  information  gathered,  including  the  interoperability  maps,  provides  the  basic  input  for 
the  SoS  Analyze  element.  To  evaluate  that  information,  the  Charting  Team  makes  use  of  the 
SoS  Framework  element  and  works  through  the  SoS  Analyze  element  to  arrive  at  a  set  of 
techniques.  Together,  those  principles  and  techniques  help  the  team  find  root  causes  and 
patterns.  The  team’s  objectives  for  the  SoS  Analyze  element  are  to 

•  characterize  enablers  and  barriers  to  interoperability  and  effective  system-of-systems 
operations 

•  identify  potential  gaps  in  the  system-of-systems  practices 

•  derive  recommendations 

•  determine  next  steps  to  initiate  the  planning  activities  of  the  SoS  Pilot  element  (e.g.,  what 
could  be  done  and  who  should  be  involved  in  actual  action  planning) 

The  activities  associated  with  the  SoS  Analyze  element  are  performed  by  the  Charting  Team, 
typically  requiring  little  or  no  interaction  with  stakeholders  of  the  system  of  systems.  The 
SoS  Analyze  element  is  composed  of  two  phases: 

1 .  data  analysis  and  findings 

2.  recommendations  and  results 

4.2.1  Data  Analysis  and  Findings  Phase 

As  the  name  implies,  the  goal  for  the  data  analysis  and  findings  phase  is  to  analyze  the 
information  collected  from  the  discussion  and  workshop  sessions,  the  interoperability  maps, 
and  other  data  sources.  When  performing  the  analysis,  the  Charting  Team  members  use  their 
knowledge  of  system-of-systems  interoperability  and  consider  the  insights  offered  in  the  SoS 
Framework  element.  The  team  analyzes  the  interoperability  maps  for  patterns  that  identify 
aspects  conducive  to  achieving  interoperability  or  indicative  of  potential  interoperability 
problems.  (See  Section  4.2.3  for  more  information  about  using  interoperability  maps.) 


CMU/SEI-2006-TN-019 


17 


The  Charting  Team  generates  a  set  of  findings  that  address,  at  a  minimum,  the 
interoperability  issues  identified  during  the  context-setting  phase  of  the  SoS  Diagnose 
element.  Often  additional  issues  surface  during  the  analysis  and  are  factored  into  the  findings. 


4.2.2  Recommendations  and  Results  Phase 

The  culmination  of  the  SoS  Analyze  element  is  the  development  of  a  set  of  recommendations 
for  improving  the  system-of-systems  effort.  These  recommendations  reflect  the  principles  of 
the  SoS  Framework  element  and  are  customized  to  the  needs  of  a  particular  system-of- 
systems  effort  and  its  constituent  parts.  Some  example  recommendations  from  our  work  with 
a  large  government  system  of  systems  are  shown  in  Table  1 . 


Table  1:  Example  Charting  Team  Recommendations 


Challenge 

Analysis 

Charting  Team 
Recommendation 

Numerous  problems  arising 
from  clashes  between 
decisions  made  by 
autonomous  constituents  (in 
this  case,  by  individual 
programs) 

The  Context  Interoperability 

Map  indicated  that  there  was  a 
high  degree  of  coupling  between 
the  various  constituents. 

The  lack  of  a  clearly  articulated, 
shared  intent  for  the  system  of 
systems  was  also  contributing  to 
the  problems. 

Definition  of  a  set  of  “guiding 
principles”:  each  constituent 
would  abide  by  those  principles 
as  part  of  operating  in  the  system 
of  systems 

The  guiding  principles  would  be 
used  to  influence  the  cross-system 
tradeoffs  and  help  individual 
program  offices  maintain 
coherence  in  their  day-to-day 
decisions. 

Limited  understanding  about 
the  nature  of  the 
interoperability  relationships 
between  constituents 

Many  of  the  relationships  were 
tacit,  implicit,  or 
unacknowledged  and  were  not 
considered  adequately  as 
decisions  were  made. 

Expansion  of  the  draft 
interoperability  maps  with  an 
initial  emphasis  on  the  schedule 
relationships  and  dependencies 
among  the  constituents 

It  is  vital  that  the  Charting  Team  communicate  its  findings  and  recommendations  effectively 
to  the  sponsoring  organization.  To  capture  and  communicate  the  results,  the  team  should 
create  a  briefing  or  short  report.  The  depth  and  breadth  of  the  results  is  based  on  the  scope 
and  expectations  negotiated  as  part  of  the  context-setting  phase  in  the  SoS  Diagnose  element. 
During  the  results  briefing,  the  Charting  Team  will  encourage  discussion  regarding  the 
accuracy  of  the  findings  and  the  feasibility  of  the  recommendations. 

Also  in  this  phase,  the  team  seeks  to  elicit  commitment  to  create  an  action  plan  that  will 
address  the  recommendations.  This  commitment  should  include  identification  of  the 
individuals  responsible  for  leading  the  action  planning  team.  While  the  actual  planning  will 
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be  performed  as  part  the  SoS  Pilot  element,  our  experience  indicates  that  it  is  critical  to  close 
out  the  SoS  Analyze  element  with  a  plan  for  initiating  the  planning  process. 

4.2.3  Using  Interoperability  Maps 

Interoperability  maps  provide  a  rich  opportunity  for  analysis.  Relationships  in  the  Node¬ 
centric  Interoperability  Maps,  for  instance,  can  suggest  possible  cascading  effects,  in  which  a 
decision  made  at  a  particular  node  has  possible  adverse  affects  on  other  nodes.  The  maps  can 
also  indicate  the  span  of  control  from  a  specific  perspective — for  example,  that  of  the 
program  manger  or  of  a  related  agency.  Further,  “islands”  or  “continents”  of  nodes  can 
indicate  elements  that  are  entirely  outside  the  span  of  control  from  the  perspective  of  a 
particular  organization.  Clusters  of  nodes  and  arcs  can  indicate  possible  bottlenecks — or 
alternately  may  provide  opportunities  for  optimization. 

Node-centric  Interoperability  Maps  provide  an  additional  level  of  detail  sufficient  to 

•  identify  interoperability  problems  with  neighbors  and  support  risk  assessment  for  the 
node 

•  allow  analysis  of  the  effect  of  changes  at  the  global  level  on  individual  nodes 

•  permit  analysis  of  the  impact  of  changes  at  the  node  level  on  related  nodes 

•  support  decisions  at  the  node  level,  consistent  with  global  objectives 

•  identify  relationships  outside  the  system-of-systems  effort  that  may  constrain  decisions  or 
affect  the  ability  to  perform  as  expected 

Arc-centric  Interoperability  Maps  provide  yet  another  level  of  analysis  to 

•  highlight  consistencies  and  inconsistencies  between  agreements  and  expectations 

•  facilitate  the  identification  and  analysis  of  alternate  possible  agreements 

•  indicate  the  possibility  of  achieving  a  goal  based  on  the  minimum  critical  needs  and 
broadest  feasible  offers  of  the  parties  involved 

•  support  impact  analysis  for  changes  in  expectations  and  intentions 
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5  Improving 


In  our  experience,  a  list  of  recommendations — even  quite  detailed  ones — is  insufficient  for 
many  organizations  to  enact  measures  and  effectively  derive  improvements.  Often  there  are 
subtleties  and  interrelationships  among  recommendations  that  are  easily  missed. 

Many  of  the  recommendations  that  come  out  of  the  SoS  Analyze  element  demand  significant 
cultural  and  process  changes.  These  types  of  recommendations  are  rarely  implemented  easily. 
While  some  organizations  may  have  existing  improvement  infrastructures  that  can  be 
leveraged,  there  is  often  little  in  place  for  cross-organization  collaboration  and  improvement. 

We  have  found  that  improvements  are  realized  more  effectively  when  recommendations  and 
their  resulting  changes  to  individuals  and  organizations  are  planned  and  managed  explicitly. 
SoS  Pilot  and  SoS  Guide,  the  Improving  elements  of  SoS  Navigator,  collectively  assist 
stakeholders  of  a  system  of  systems  to 

•  plan,  execute,  and  manage  the  implementation  of  recommendations 

•  prototype  and  validate  new  approaches  within  the  system-of-systems  environment 

•  perform  necessary  adaptations  for  the  specific  environment 

•  determine  viable  rollout  strategies  to  the  broader  set  of  organizations 

•  monitor  the  effectiveness  of  the  new  approaches 

For  a  given  system  of  systems,  the  Improving  Team  (ideally  containing  at  least  one  member 
from  the  Charting  Team,  for  greater  efficiency  and  continuity)  will  typically  iterate  between 
the  activities  of  the  SoS  Pilot  and  SoS  Guide  elements  to  enact  the  recommendations  from 
the  SoS  Analyze  element. 


5.1  SoS  Pilot 

The  SoS  Pilot  element  is  a  set  of  techniques  for  applying  existing  improvement  approaches 
where  appropriate  and  adapting  them  as  needed  in  multiorganizational  interactions.  It  builds 
on  the  organization-specific  information,  findings,  and  recommendations  of  the  SoS  Analyze 
element  and  the  paradigms,  practices,  and  techniques  of  the  SoS  Framework  element. 

The  four  primary  objectives  for  the  SoS  Pilot  element  are  to  assist  organizational  entities  to 

1.  form  a  new  improvement  roadmap  or  revise  an  existing  one  for  a  specific  system  of 
systems 

2.  demonstrate,  prototype,  and  pilot  the  use  of  new  system-of-systems  practices  in  the 
specific  system-of-systems  context 
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3.  adapt  system-of-systems  practices  as  necessary  for  the  local  environment  without 
compromising  the  underlying  practice  or  technique 

4.  gain  commitment  to  proceed  further 

In  the  SoS  Pilot  element,  the  Improving  Team  works  with  the  various  system-of-systems 
entities  (i.e.,  global  system-of-systems  entity,  autonomous  constituents,  COIs,  and 
neighborhoods)  to  create  (or  refine)  an  implementation  roadmap  that  includes  a  set  of 
detailed  action  plans. 

The  degree  of  formality  of  the  plans  depends  on  the  context  (political  and  cultural 
environment)  of  the  particular  system  of  systems  and  the  unbounded  nature  of  systems  of 
systems  generally.  It  is  impractical  for  any  one  entity  to  have  complete  visibility  and  exercise 
complete  control.  Because  of  these  constraints,  the  goal  is  to  find  the  degree  of  formality 
required  to  be  effective,  but  no  more — an  objective  quite  different  from  the  one  many  are 
accustomed  to  in  a  traditional  monolithic  system  environment. 

The  Improving  Team  works  with  the  various  system-of-systems  entities  to  set  up  any 
necessary  improvement  infrastructures  or  adapt  existing  structures  to  align  with  the  core 
paradigms,  principles,  and  practices  reflected  in  the  SoS  Framework  element.  An  essential 
aspect  of  the  team’s  work  is  identifying  a  System-of-Systems  Practices  Group  of  interested 
individuals  who  can  be  led  toward  an  understanding  of  the  system-of-systems  practices  and 
can  assume  ownership  of  the  improvement  activities. 

Another  critical  part  of  the  SoS  Pilot  element  is  to  identify  and  implement  a  prototype  or 
demonstration  project  for  the  highest  priority  recommendations  and  action  plans.  In  that 
project,  the  team’s  goal  is  to  demonstrate  the  use  of  selected  system-of-systems  practices  in  a 
pilot  setting.  Initially,  the  Improving  Team  members  act  as  mentors  for  the  project,  which 
provides  an  excellent  opportunity  to  involve  the  System-of-Systems  Practices  Group.  The 
following  section  outlines  how  the  ISIS  team  is  using  the  SoS  Pilot  element  with  a  large 
government  system  of  systems. 

5.1.1  SoS  Pilot  in  Use:  Activities 

In  work  with  a  large  system  of  systems,  one  of  the  initial  steps  taken  by  the  Improving  Team 
was  to  work  with  the  global  system-of-systems  entity  to  identify  and  establish  a  System-of- 
Systems  Practices  Group.  The  initial  members  of  this  group  came  from  several  of  the 
constituents  of  the  system  of  systems.  This  set  of  constituents  was  involved  in  some  of  the 
most  critical  interoperability  problems  identified  in  the  SoS  Analyze  element.  A  charter  was 
developed  by  the  group  and  accepted  by  the  global  and  constituent  entities  of  the  system  of 
systems.  While  in  time  additional  improvement  infrastructure  may  be  advisable,  this  charter 
was  deemed  sufficient  for  a  first  iteration. 

The  System-of-Systems  Practices  Group  took  the  recommendations  from  the  SoS  Analyze 
element,  prioritized  them,  and  decided  to  focus  initially  on  understanding  and  managing 
influence  relationships  among  constituents.  The  group’s  action  plan  called  for  a  small 
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demonstration  project  to  experiment  with  a  set  of  interoperability  maps,  placing  an  initial 
emphasis  on  the  schedule  relationships  and  dependencies  among  a  subset  of  the  system-of- 
systems  constituents. 

The  Improving  Team  worked  with  the  System-of- Systems  Practices  Group  to  define  the 
purpose  and  scope  for  the  demonstration  project.  The  stated  purpose  was  to  provide  early 
warning  for  interoperability  issues  that  otherwise  might  not  have  been  identified  until  too 
late — that  is,  to  identify  solutions  before  potential  problems  became  actual  problems.  Further, 
the  group  wanted  to  determine  whether  its  approach  could  be  applied  on  a  broader  basis.  The 
demonstration  project  would  be  constrained  to  a  limited  number  of  constituents;  yet,  it  was 
sufficiently  rich  to  examine  a  number  of  cross-program  relationships.  The  duration  of  the 
demonstration  project  was  constrained  to  an  8-  to  10-week  period. 

Activities  during  the  demonstration  project  included  refining  an  initial  Context 
Interoperability  Map  (from  the  SoS  Analyze  element),  creating  Node-centric  Interoperability 
Maps,  and  creating  a  corresponding  set  of  Arc-centric  Interoperability  Maps.  The  Improving 
Team  worked  with  the  System-of- Systems  Practices  Group  to  understand  and  apply  the 
interoperability  maps  and  other  applicable  aspects  of  the  SoS  Framework  element. 

5.1.2  SoS  Pilot  in  Use:  Results 

From  the  pilot,  we  learned  that  interoperability  maps  should  be  developed  incrementally. 
These  maps  should  be  updated  as  more  is  known  about  context,  the  individual  nodes,  and 
relationships.  In  general,  we  suggest  that  organizations 

•  establish  a  few  important  but  easily  understood  attributes  (e.g.,  schedule) 

•  begin  with  broadest  identification  of  direct  and  indirect  influences  of  some  critical 
decision,  rather  than  direct  effects  of  many  decisions 

•  have  some  early  successes  and  learning  before  adding  detail  and  more  cases 

•  start  with  simple  tools  such  as  word  processors  and  spreadsheets  (Investing  in  more 
sophisticated  tools  should  be  explored  during  SoS  Guide,  after  more  experience  has  been 
gained.) 

With  assistance  from  the  Improving  Team  and  using  information  from  the  demonstration 
project,  the  System-of- Systems  Practices  Group  analyzed  the  interoperability  maps  for 
patterns  that  could  be  symptomatic  of  interoperability  problems.  Several  potential  problems 
were,  in  fact,  identified.  (The  project  results  and  recommendations  had  not  been  presented  to 
the  global  system-of-systems  entity  and  key  constituents  at  the  time  of  the  publishing  of  this 
technical  note.) 
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5.2  SoS  Guide 


The  SoS  Guide  element  is  a  set  of  techniques  geared  to  expanding  the  use  of  the  practices 
demonstrated  in  the  SoS  Pilot  element  to  the  entire  range  of  organizations  involved  in  a 
system-of-systems.  The  primary  objectives  for  the  SoS  Guide  element  are 

•  describing  readiness  for  the  cultural  and  organizational  assumptions  of  the  SoS 
Framework  element 

•  characterizing  the  adoption  risks 

•  forming  and  executing  an  institutionalization  plan  for  the  broader  COI 

•  creating  or  expanding  the  change  infrastructure  across  the  broader  community 

•  monitoring  the  effectiveness  of  adopted  practices  and  recommending  adjustments  as 
necessary 

The  SoS  Guide  element  leverages  traditional  improvement  strategies,  where  applicable. 
Flowever,  work  is  needed  to  expand  these  strategies  to  have  greater  effectiveness  in  the  large 
multiorganization  environments  of  systems  of  systems. 
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6  Summary 


SoS  Navigator  is  a  set  of  integrated  practices  that  address  the  challenges  related  to  achieving 
effective  interoperability  in  a  large  system-of-systems  context.  The  constituents  of  such 
systems  of  systems  are  highly  dynamic  and  involve  diverse  stakeholders.  These  systems  of 
systems  often  evolve  into  existence  and  continue  to  evolve  throughout  their  life  cycles — as 
new  constituents  are  built,  existing  systems  connect  to  become  constituents,  and  other 
constituents  leave.  Lines  between  development,  acquisition,  and  operation  cycles  are 
increasingly  blurred. 

SoS  Navigator  helps  organizations  chart  a  technical,  organizational,  and  operational  path 
through  this  system-of-systems  environment  and  prepare  for  the  even  more  demanding 
environments  of  the  future,  Internet-scale  systems  of  systems.  SoS  Navigator  consists  of  a 
core  set  paradigms  and  principles,  along  with  techniques  to  identify  and  improve  the 
practices  of  organizations. 

SoS  Navigator  was  developed  to  provide  concrete  guidance  to  programs  and  organizations 
that  are  addressing  the  new  realities  of  system-of-systems  development  and  acquisition.  SoS 
Navigator  is  based  on  real  engagements  with  actual  large-scale  systems  of  systems.  It  has 
been  used  successfully  to  provide  insights  that  had  not  been  previously  understood;  those 
insights  have  been  applied  to  avoid  potentially  expensive  problems. 

SoS  Navigator  is  an  evolving  product.  As  organizations  like  FCS,  USJFCOM,  the  medical 
community,  and  others  gain  expertise  in  building  ever-larger  systems  of  systems,  the  ISIS 
team  intends  to  incorporate  the  improved  and  new  practices  they  identify  into  SoS  Navigator. 
Likewise,  SoS  Navigator  will  evolve  as  researchers  identify  new  approaches  in  architecting, 
constructing,  maintaining,  and  operating  large-scale  systems  of  systems. 

The  ISIS  team  welcomes  your  feedback  on  this  discussion  of  the  SoS  Navigator  technology 
and  comments  on  your  experiences  with  interoperability  challenges.  Contact  the  team  at  isis- 
sei@sei.cmu.edu. 
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